Optimal extraction of data contained within an electromagnetic signal requires the removal of defects such as noise and instrument limitations. A key area in which optimized extraction and reconstruction of data is sought is the field of image enhancement. Even when the instruments can be made essentially noise-free, instrumental signatures related to finite spatial, spectral, or temporal resolution remain. At this point, image reconstruction is required to remove the instrumental signatures Applications of image enhancement, and the sources of noise and other factors that can negatively impact data extraction efficiency cover a wide range including astronomical observation and planetary exploration, where sources can be faint and atmospheric interference introduce noise and distortion, military and security surveillance, where light can be low and rapid movement of targets result in low contrast and blur, medical imaging, which often suffers from lack of clarity, and video images, where transmission and instrument limitations, and the need for real time response, can negatively impact image sharpness and detail.
Images degraded by imperfect sensors or transmission can often be modeled as the sum of random image noise and a convolution of the true image with a point response function (PRF) or blurring kernel:D(i)=∫dyH(y,i)I(y)+N(i),  (1)where D(i) is the data in cell i (typically a pixel), I is the image model, H is the point-response-function (PRF) due to instrumental and possible atmospheric blurring. The PRF is often only a function of displacement between pixels. In general, the PRF can vary across the field.
Image reconstruction differs from standard solutions of integral equations due to the noise term, N, the nature of which is only known statistically. Methods for solving such an equation fall under the categories of (1) direct methods, which apply explicit operators to data to provide estimates of the image, and (2) indirect methods which model the noiseless image, transform it forward to provide a noise-free data model, then fit the parameters of the image to minimize the residuals between the real data and the noise-free data model. The direct methods have the advantage of speed, but they tend to amplify noise, particularly at high spatial frequencies. The indirect methods supposedly exclude the noise, however, the requirement of imposing a model can be a disadvantage. If a good parametric form for the image is known a priori, the result can be very good.
Existing indirect methods of image enhancement such as chi-squared fitting and maximum entropy often result in poor quality images, introduce artifacts, and operate at speeds that are too slow for video. In spite of the inadequacies of current image enhancement techniques, the market for real time video image enhancement is growing rapidly. For example, in the U.S., military applications of video imaging for detailed reconnaissance and remote sensing information has increased over the past decade, despite a reduction in overall defense spending. The increased reliance on night-vision and heat-sensing video will only increase the demand for image enhancing devices. In the medical diagnostic field, growing restrictions imposed by insurance providers has restricted the use of sophisticated high cost imaging machinery, leading to a desire to enhance more established and less expensive imaging methods to improve their sensitivity and expand their usefulness. Furthermore, even the more sophisticated medical imaging devices, e.g., nuclear imaging methods, often suffer from noise and lack of contrast which can make diagnosis difficult, such that image enhancement is needed to optimize the imaging system's capabilities.
In imaging, the input data in many cases is obtained with a CCD detector array with a particular pixel size and shape. For example, in the imaging of star fields, the image would be best represented by a sum of point sources with arbitrarily precise positions and brightness. Since large regions of the data field will have few, if any, photon counts, portions of the pixel grid are unused, and the degrees of freedom, i.e., pixels, representing these portion of the image over-specify the data. In other portions of the image, the density of pixels may be too sparse to adequately represent the image.
The Pixon™ methods disclosed in U.S. Pat. No. 5,912,993 and co-pending application Ser. No. 09/333,172, the disclosures of which are incorporated herein by reference, approach the absolute limit of image improvement, cleanly extracting the entire image information content. In these methods, the sizes and shapes of the Pixon™ kernels are adjusted to decompose the image into the smallest number of Pixon™ elements, where each element encompasses multiple pixels. Improvement over the competing methods, some of which are described above, reaches a factor of 10 to 100 improvement in sensitivity and a factor of a few in resolution, together with robust rejection of noise and spurious artifacts. However, the previously-described Pixon™ methods achieve minimum complexity by selecting a set of Pixon™ kernels using an iterative method to optimize the size and shape of the kernels. While such an iterative process is highly effective for enhancement and reconstruction of still images, video enhancement requires real-time or nearly real-time response, making methods with a large number of iterations impractical.
Accordingly, the need remains for a method of image reconstruction which is capable of video rate response which can be readily implemented in hardware. The invention disclosed herein provides such a method.